Papers with word embedding models
Using Graphs for Word Embedding with Enhanced Semantic Relations (D19-53)
Copied to clipboard
| Challenge: | Word embedding algorithms are based on sequential text input, while others are utilizing a graph representation of text. |
| Approach: | They propose a word embedding algorithm based on a directed word graph to provide additional information for sequential text input algorithms. |
| Outcome: | The proposed algorithm is based on a directed word graph to provide additional information for sequential text input algorithms. |
Igbo Diacritic Restoration using Embedding Models (N18-4)
Copied to clipboard
| Challenge: | Igbo is a low-resource language spoken by approximately 30 million people worldwide. |
| Approach: | They propose to use word embeddings to restore diacritics in Igbo by using a pre-processing task that replaces missing diacrittics on words from which they have been removed. |
| Outcome: | The embedding models performed better than n-gram models on the diacritic restoration task. |
SanskritShala: A Neural Sanskrit NLP Toolkit with Web-Based Interface for Pedagogical and Annotation Purposes (2023.acl-demo)
Copied to clipboard
| Challenge: | SanskritShala is a neural-based Sanskrit NLP toolkit that is available as a web-based application . |
| Approach: | They propose a neural Sanskrit NLP toolkit that facilitates linguistic analyses for word segmentation, morphological tagging, dependency parsing, and compound type identification. |
| Outcome: | The proposed toolkit reports state-of-the-art performance on benchmark datasets . it is built with easy-to-use interactive data annotation features . |
Representing ELMo embeddings as two-dimensional text online (2021.eacl-demos)
Copied to clipboard
| Challenge: | ELMoViz module adds support for contextualized embedding architectures, in particular for token embeddable word models. |
| Approach: | They propose to add a module to the free and open-source WebVectors toolkit which provides lexical hyperlinks to word representations in static embedding models. |
| Outcome: | The ELMoViz module adds support for contextualized embedding architectures, in particular for ELMa models. |
Enriching Word Embeddings with Domain Knowledge for Readability Assessment (C18-1)
Copied to clipboard
| Challenge: | Existing word embedding models focus on syntactic or semantic relations of words, while ignoring reading difficulty. |
| Approach: | They propose a method which learns the word embedding for readability assessment . they extract the knowledge on word-level difficulty from three perspectives to construct a knowledge graph . |
| Outcome: | The proposed method is effective and potential, the authors show . they use the knowledge-enriched word embedding model on English and Chinese datasets . |
Additive Compositionality of Word Vectors (D19-55)
Copied to clipboard
| Challenge: | Existing research on justifying additive compositionality of word embedding models requires a rather strong assumption of uniform word distribution. |
| Approach: | They propose to relax the assumption of uniform word distribution and propose more realistic conditions for proving additive compositionality. |
| Outcome: | The proposed model improves on word similarity and noisy sentence similarity. |
Lipstick on a Pig: Debiasing Methods Cover up Systematic Gender Biases in Word Embeddings But do not Remove Them (N19-1)
Copied to clipboard
| Challenge: | Existing methods to remove gender bias from word embeddings are insufficient, we argue . existing methods for gender-neutral modeling are ineffective, we conclude . |
| Approach: | They propose methods to reduce gender bias in word embeddings by debiasing them using text corpora. |
| Outcome: | The proposed methods show that they can reduce gender bias in word embeddings . the proposed methods are insufficient and should not be trusted, the authors argue . |
Leveraging Meta-Embeddings for Bilingual Lexicon Extraction from Specialized Comparable Corpora (C18-1)
Copied to clipboard
| Challenge: | Recent studies on bilingual lexicon extraction from specialized comparable corpora show differences in performance . lack of large specialized corporan to build efficient representations can be partially explained . |
| Approach: | They propose to use character-based embedding models to combine different embeddable models . they emphasize how character-driven embeddance models outperform other models on quality . |
| Outcome: | The proposed model outperforms other models on quality of extracted bilingual lexicons . comparable corpora are an interesting and practical alternative to parallel corporation . |
Diachronic word embeddings and semantic shifts: a survey (C18-1)
Copied to clipboard
| Challenge: | Existing methods for tracing time-related semantic shifts with word embedding models lack the cohesion, common terminology and shared practices of more established areas of natural language processing. |
| Approach: | They propose several axes along which these methods can be compared and propose a framework for comparison. |
| Outcome: | The proposed methods are compared with existing methods and outline their main challenges and potential applications. |
Identification of Indigenous Knowledge Concepts through Semantic Networks, Spelling Tools and Word Embeddings (2020.lrec-1)
Copied to clipboard
| Challenge: | Using semantic tools and network methods, dialectal variations of words are typically used to access native, regional knowledge contained in language corpora. |
| Approach: | They propose a technique for identifying dialectal variations of words in a non-standard language legacy dataset using GermaLemma. |
| Outcome: | The proposed method may be used in other projects dealing with non-standard or endangered language collections, aiming to access, analyze and ultimately preserve native regional knowledge. |
Evaluating Word Embeddings with Categorical Modularity (2021.findings-acl)
Copied to clipboard
| Challenge: | Existing word embeddings use different bilingual supervision signals with varying levels of strength. |
| Approach: | They propose a graph modularity metric to measure word embedding quality . they use a set of 500 words belonging to 59 neurobiologically motivated semantic categories . |
| Outcome: | The proposed metric measures word embedding quality on monolingual and cross-lingual tasks. |
Evaluation of Sentence Representations in Polish (2020.lrec-1)
Copied to clipboard
| Challenge: | Existing methods for learning sentence representations have been limited in low-resource languages such as Polish . |
| Approach: | They propose two new Polish datasets for evaluating sentence embeddings and evaluate eight different methods including Polish and multilingual models. |
| Outcome: | The proposed methods show strengths and weaknesses in Polish and multilingual models. |
Indra: A Word Embedding and Semantic Relatedness Server (L18-1)
Copied to clipboard
Juliano Efson Sales, Leonardo Souza, Siamak Barzegar, Brian Davis, André Freitas, Siegfried Handschuh
| Challenge: | Word embedding/distributional semantic models are a fundamental component in many natural language processing (NLP) architectures. |
| Approach: | They propose a multi-lingual word embedding/distributional semantics framework which supports creation, use and evaluation of word embedded models. |
| Outcome: | The proposed tool supports the creation, use and evaluation of word embedding models. |
Word Embedding Evaluation for Sinhala (2020.lrec-1)
Copied to clipboard
| Challenge: | Word embeddings are a key component of the natural language processing process. |
| Approach: | They evaluate three standard word embedding models for Sinhala language using two evaluation methods: intrinsic evaluation and extrinsic evaluation. |
| Outcome: | The proposed models performed best in the three evaluation tasks, while FastText and Glove showed the lowest accuracies. |
BERT is to NLP what AlexNet is to CV: Can Pre-Trained Language Models Identify Analogies? (2021.acl-long)
Copied to clipboard
| Challenge: | Analogies play a central role in human commonsense reasoning. |
| Approach: | They analyze the capabilities of transformer-based language models on an unsupervised task . they find off-the-shelf language models can identify analogies to a certain extent . |
| Outcome: | The proposed language models outperform word embedding models on an unsupervised task . the best results were obtained with GPT-2 and RoBERTa . |
Manifold Learning-based Word Representation Refinement Incorporating Global and Local Information (2020.coling-main)
Copied to clipboard
| Challenge: | Recent studies show word embedding models underestimate similarities between similar words and overestimate similarities between distant words. |
| Approach: | They propose two new word embedding methods that align original and re-fined embeddable spaces to a new refined semantic space. |
| Outcome: | The proposed methods outperform state-of-the-art methods for word representation refinement. |
Word and Document Embedding with vMF-Mixture Priors on Context Word Vectors (P19-1)
Copied to clipboard
| Challenge: | Word embedding models typically learn two types of vectors: target word vectors and context word vector. |
| Approach: | They propose to explicitly impose a cluster structure on context word vectors to improve word embedding models. |
| Outcome: | The proposed model improves word embedding models qualitatively by imposing a cluster structure on the set of context word vectors. |
SENCORPUS: A French-Wolof Parallel Corpus (2020.lrec-1)
Copied to clipboard
| Challenge: | Currently, the corpus is being used to develop a neural machine translation model to translate French sentences into Wolof. |
| Approach: | They propose to build a bilingual parallel corpus between French and Wolof, a Niger-Congo language belonging to the Northern branch of the Atlantic group. |
| Outcome: | The proposed corpus contains 70,000 parallel sentences drawn on various sources from different domains. |
Methods for Numeracy-Preserving Word Embeddings (2020.emnlp-main)
Copied to clipboard
Dhanasekar Sundararaman, Shijing Si, Vivek Subramanian, Guoyin Wang, Devamanyu Hazarika, Lawrence Carin
| Challenge: | Word embedding models capture semantic relationships between words but fail to capture numerical properties associated with numbers. |
| Approach: | They propose a method to assign and learn embeddings for numbers using word embedders. |
| Outcome: | The proposed model outperforms pre-trained word embedding models across multiple examples of two tasks. |
Profiling of Intertextuality in Latin Literature Using Word Embeddings (2021.naacl-main)
Copied to clipboard
| Challenge: | a new study examines the intertextual relationships between authors in classical Latin literature . a large corpus of lemmatized Latin is used to train word embeddings . |
| Approach: | They propose to train an optimized word2vec model on a large corpus of Latin . they then replicate a previous study of the Roman historian Livy using hand-crafted stylometric features. |
| Outcome: | The proposed model outperforms a widely used lexical search method on Latin epic poetry . it advances the development of core computational resources for a major premodern language . |
Aligning Multidimensional Worldviews and Discovering Ideological Differences (2021.emnlp-main)
Copied to clipboard
| Challenge: | Existing work on understanding worldviews and ideological distinctions focuses on political polarization . et al., 2018: a novel method for uncovering complex ideological and worldview characteristics of communities. |
| Approach: | They propose a method to uncover multifaceted ideological differences across multiple axes . they use comments from the largest communities on reddit.com to train word embedding models . |
| Outcome: | The proposed method can uncover complex ideological differences across multiple axes of polarization using over 1B comments from the largest communities on reddit.com representing 40% of Reddit activity. |
Self-Attentive, Multi-Context One-Class Classification for Unsupervised Anomaly Detection on Text (P19-1)
Copied to clipboard
| Challenge: | Existing methods for unsupervised anomaly detection use pre-trained word embeddings . proper text representation is critical for designing well-performing machine learning algorithms . |
| Approach: | They propose a new anomaly detection method that builds upon word embedding models to learn multiple sentence representations that capture multiple semantic contexts via the self-attention mechanism. |
| Outcome: | The proposed method performs on Reuters, 20 Newsgroups, and IMDB Movie Reviews datasets. |
Facilitating Corpus Usage: Making Icelandic Corpora More Accessible for Researchers and Language Users (2020.lrec-1)
Copied to clipboard
| Challenge: | Gigaword corpus is a large text corpus used in natural language processing . large corpora are needed to achieve better performance in the field of NLP . |
| Approach: | They propose a set of tools to facilitate the use of the Icelandic Gigaword Corpus . they provide n-grams based on the corpus, and a variety of pre-trained word embeddings models . |
| Outcome: | The proposed tools facilitate the use of the Icelandic Gigaword corpus in the field of Natural Language Processing and other fields. |
IceBATS: An Icelandic Adaptation of the Bigger Analogy Test Set (2022.lrec-1)
Copied to clipboard
| Challenge: | a new test set that measures word embeddings' ability to recognize linguistic regularities is presented in a paper in elijsson, iran . the test sets are a good quality estimator for extrinsic evaluation of word embedded models . |
| Approach: | They propose a test set that measures language models' ability to recognize linguistic regularities in a balanced way. |
| Outcome: | The proposed set is apt at measuring the capabilities of word embedding models. |
Statistical Uncertainty in Word Embeddings: GloVe-V (2024.emnlp-main)
Copied to clipboard
| Challenge: | Static word embeddings are ubiquitous in computational social science applications . however, assessing the statistical uncertainty in downstream conclusions remains challenging . |
| Approach: | They propose a method to obtain approximate, easy-to-use, and scalable reconstruction error variance estimates for one of the most widely used word embedding models. |
| Outcome: | The proposed method enables hypothesis testing in key word embedding tasks. |
Enhancing Deep Learning with Embedded Features for Arabic Named Entity Recognition (2022.lrec-1)
Copied to clipboard
| Challenge: | Word embeddings can capture the semantics of words and other hidden features, but the Arabic language is complex and requires a large amount of information to process. |
| Approach: | They propose to add morphological and syntactical features to Arabic word embeddings to train the model. |
| Outcome: | The proposed model outperforms the previous systems to the best of our knowledge. |
Conditional Word Embedding and Hypothesis Testing via Bayes-by-Backprop (D18-1)
Copied to clipboard
| Challenge: | Whether word's meaning varies across contexts has become a major focus of research in recent years. |
| Approach: | They propose a word embedding model that incorporates document covariates to estimate conditional word embeds. |
| Outcome: | The proposed model estimates word embedding distributions based on document covariates . if word embeds are statistically significant, hypothesis tests can be performed . |
GGP: Glossary Guided Post-processing for Word Embedding Learning (2020.lrec-1)
Copied to clipboard
| Challenge: | Existing word embedding models require much training time and domain knowledge to improve. |
| Approach: | They propose a GGP-based word embedding model that incorporates the glossary and learns sense representations. |
| Outcome: | The proposed model outperforms existing models on topical/functional similarity datasets by 4.1% and 7%. |
Towards a Gold Standard for Evaluating Danish Word Embeddings (2020.lrec-1)
Copied to clipboard
| Challenge: | Existing word embedding models resemble semantic similarity solely by distribution, but there seems to be a need for future judgments to measure similarity in full context and along more than a single spectrum. |
| Approach: | They propose a model-agnostic similarity goal standard for evaluating Danish word embeddings based on human judgments made by 42 native speakers of Danish. |
| Outcome: | The goal standard is applied to evaluate Danish word embeddings on 42 native speakers of Danish. |
Attention Word Embedding (2020.coling-main)
Copied to clipboard
| Challenge: | Word embedding models learn semantically rich vector representations of words . popular word embedders include word2vec, GloVe, and fastText . |
| Approach: | They propose an AWE-S model which integrates the attention mechanism into the CBOW model and incorporates subword information. |
| Outcome: | The proposed model outperforms the state-of-the-art model on word similarity datasets and when used for initialization of NLP models. |
Unsupervised Attention-based Sentence-Level Meta-Embeddings from Contextualised Language Models (2022.lrec-1)
Copied to clipboard
| Challenge: | Existing methods for creating metaembeddings from static word embeddings have been proposed, but they are not tied to a particular downstream task. |
| Approach: | They propose a sentence-level meta-embedding learning method that takes contextualised word embedding models and learns a phrase embeddable that preserves complementary strengths of the input source NLMs. |
| Outcome: | The proposed method outperforms existing methods on semantic textual similarity benchmarks on a supervised baseline and on token-level embeddings. |
To Revise or Not to Revise: Learning to Detect Improvable Claims for Argumentative Writing Support (2023.acl-long)
Copied to clipboard
| Challenge: | assessing whether and how different claims in a text need to be revised is a hard task, especially for novice writers. |
| Approach: | They propose a sampling strategy based on revision distance to capture differences between versions of the same text. |
| Outcome: | The proposed sampling strategy can be done without additional annotations and judgments. |